Color rendering

ABSTRACT

This document discloses a method, system, and/or software configured to provide improved color rendering of an item or scene, such as an item for sale through web sites or printed brochures (e.g., color catalogs). This improved color rendering can be provided through improved accuracy when rendering an online item, such as on a user&#39;s laptop, smartphone, or desktop computer showing a webpage. Further still, this document discloses ways in which to solve numerous problems in the field of online sales where an accurate rendering of an item, and even a customized rendering of an item to fit a buyer&#39;s intended use for the item, is desired. This document also describes ways in which to improve color renderings for a print item through use of a user&#39;s device to capture an image using the device&#39;s camera, and then display the item more accurately or congruently than it was originally displayed in print.

RELATED APPLICATIONS

The present disclosure claims priority to U.S. Provisional PatentApplication Ser. No. 62/964,571, filed Jan. 22, 2020, the disclosure ofwhich is incorporated by reference herein in its entirety.

BACKGROUND

Viewing items online, or even through mailed, paper catalogs, suffersfrom inaccurate renderings of the items.

Consider, for example, an off-white chair for sale in a paper catalog.Should a buyer wish to purchase this chair in the off-white color, he orshe may lack confidence that the color matches his or her currentoff-white décor and then refuse to buy it, costing the seller the profitand the buyer the benefit of ownership. Or, even worse, the buyer maypurchase the chair believing that it matches his or her décor, only todiscover on delivery that it does not. Then, the buyer or the sellermust pay for the shipping to return the item, and both must take thetime and effort to handle the return. These are expensive failurescaused by inaccurate color renderings of items in catalogs and similarpaper renderings.

Inaccurate renderings (e.g., image depictions of an item) are oftenworse in online sales than those in catalogs. The color renderings areoften inaccurate, as they rely on image-capture and image-renderingdevices, each of which may introduce color errors. These inaccuraterenderings often show shades slightly or drastically different from theactual color of the item. Returns are common on this basis alone,causing innumerable losses in shipping the item both to and from thebuyer, irritation of the buyer, and loss of time and money for both thebuyer and seller.

SUMMARY

This document discloses a method, system, and/or software configured toprovide improved color rendering of an item or scene, such as an itemfor sale through websites or printed brochures (e.g., color catalogs).This improved color rendering can be provided through improved accuracywhen rendering an online item, such as on a user's laptop, smartphone,or desktop computer showing a webpage. Further still, this documentdiscloses ways in which to solve numerous problems in the field ofonline sales where an accurate rendering of an item, and even acustomized rendering of an item to fit a buyer's intended use for theitem, is desired. This document also describes ways in which to improvecolor renderings for a print item through use of a user's device tocapture an image using the device's camera, and then display the itemmore accurately or congruently than it was originally displayed inprint.

BRIEF DESCRIPTION OF THE DRAWINGS

The application file contains at least one drawing executed in color.Copies of this application with color drawing(s) will be provided by theOffice upon request and payment of the necessary fee. Aspects oftechniques and devices enabling refined search with machine learning aredescribed with reference to the following drawings. The same numbers areused throughout the drawings to reference like features and components:

FIG. 1 illustrates an example environment in which techniques for colorrendering can be implemented.

FIG. 2 illustrates an example user interface having an image of an itemand physical color standards.

FIG. 3 illustrates an initial set of the slider-based controls and thentwo successive user selections and resulting alterations to an item andphysical color standard.

FIG. 4 illustrates an example method by which the techniques improve anaccuracy or congruity of an item or scene.

FIG. 5 illustrates a range of different brightness choices selectablethough a user interface.

FIG. 6 illustrates a range of nine different hues and levels ofbrightness presented through a user interface.

FIG. 7 illustrates an example method by which the techniques improve,through color matching, an accuracy or congruity of a current or futurerendering of an item or scene.

FIG. 8 illustrates an item, a physical color standard captured with ornear the item, and an accurate or corrected color standard having aknown color and location.

FIG. 9 illustrates an example method by which the techniques improve arendering of an item or scene to be more accurate or congruent.

FIG. 10 illustrates an augment-reality interface presenting an image ofan item and physical color standard, and showing a user's locale.

FIG. 11 illustrates an example method by which the techniques enable aseller to provide a more-accurately rendered image of an item or scene,such as one for sale.

FIG. 12 illustrates an example method by which the techniques enable abuyer or other viewer to more-accurately or more-congruently render animage having an electronic format (email, text, social media, webpage,etc.) or a printed item or scene.

DETAILED DESCRIPTION

Overview

This document discloses a method, system, and/or software configured toprovide improved color rendering. Examples include photographs, such asfamily members captured with a camera, items for sale or presentationthrough web sites, or items for sale or presentation in printedbrochures, such as color catalogs. The techniques enable improved colorrendering through a variety of manners, which improve accuracy whenrendering an online item, such as a user's laptop, smartphone, ordesktop computer showing a webpage. This document also discloses ways inwhich to solve numerous problems in the field of online rendering wherean accurate depiction of an item or scene is desired. Further still, thedisclosed techniques can enable a customized rendering of an item to fita buyer's intended use for the item. This document also describes waysin which to improve a depiction of an item in a printed image throughuse of a user's device to capture a picture of the printed image usingthe device's camera, and then display the item more accurately on thedevice's display than it was originally displayed in print.

Inaccurate Original Image Capture

An original image capture of an item or scene can be inaccurate. Thus,well before display of the item on a website for sale or even providingthe image of the item to be printed, the image captured for the item canbe inaccurate. The techniques disclosed here improve the accuracythrough use of a physical color standard. This physical color standardprovides a check on the color of the captured image. To make use of thisphysical color standard, an image is captured of the item along with thephysical color standard. This can be done with the physical colorstandard within the image taken of the item, taken contemporaneously, ortaken in a same or similar ambient condition. The provider of the image,such as a seller, can use the physical color standard to alter thecaptured image by comparing the displayed physical color standard withhis or her own visual perception of the physical color standard. Thiscan be done by a person, which introduces some possible error but alsomay correct for errors as well as differences between light that isactually reflected from an object (the actual, measurable physics of thereflected light) and what a human perceives.

The physical color standard is capable of being copied or standardized,and in fact can be the same actual physical color standard as imagedwith the item, or it can be captured with the aid of a computer programthat stores a rendering and data concerning the physical color standard.In either or both cases, use of the physical color standard can improvethe accuracy of the image of the item or scene. The entity capturing theimage, such as a seller, can then provide the altered, improved imagefor the webpage or to a printer. This disclosed technique improves theaccuracy of the depiction of the item or scene, which on its own mayimprove a future rendering of the item as it is eventually rendered toanother entity, such as a viewer of the scene or a buyer of the item.

Furthermore, through use of the physical color standard, the techniquescan correct an image of an item or scene automatically through use ofrecorded color information for the physical color standard. Thesetechniques are further described herein.

Inaccurate or Incongruent Rendering

There are numerous ways in which a color image may be inaccurate when itis rendered on paper or a display. As noted, this can start with aninaccurate image capture of the item or scene, but this is far from theonly problem.

Consider a case where a seller captures an image with a camera that doesnot accurately capture the item's color or in ambient light that causesthe item to look differently colored than with typical ambient light,such as atypical lights in a home, ambient light from the sun when theimage was captured in a fluorescent-lit studio, or captured indoors whenthe item is likely to be used outdoors.

In these cases, the original image is not congruent to what the itemwould likely look like in another ambient condition, such as the buyer'sintended placement for the item, e.g., outdoors for outdoor furniture,clothing for indoors and outdoors, and indoors for a painting, pillows,carpet, and so forth. As noted, the original image may further beinaccurate on its face, without the error being caused by lightingdifferences.

To correct this flaw, the techniques disclosed here can provide aphysical color standard. This physical color standard provides a checkon the color of a rendered image either contemporaneous with therendering of the image or through prior calibration (described ingreater detail below). To make use of this physical color standard, acapturing entity, such as a seller, captures the image of the item alongwith the physical color standard. As noted, this can be done with thephysical color standard within the image taken of the item (though itcan also be excised from the image and provided on request in thewebpage so as not to clutter the item's spacing until a physical colorstandard rendering is requested).

In more detail, consider a shirt of a light blue-green color. Theshirt's color is likely of importance to a buyer. Thus, rather thanguess if the seller took an accurate or congruent image of the shirt, oreven guess if the buyer's own display is accurate for the colors beingrendered, the seller captures the image of the shirt with the physicalcolor standard. Then, additional opportunities arise for accuraterendering for the buyer. Note that the term accurate reflects anaccurate depiction of a color of an item as the image is rendered,relative to how the item looks in real-life at the time of capture.Congruent, however, represents how the item would look in the samelocale or another locale different from the locale in which the imagewas originally captured. Thus, a user may desire accuracy or congruency,and sometimes a blend of both. When buying an item, buyers desire toreceive an image of the item in a catalog or website that is accurate.It is also desirable for the image of the item to look as it would inthe locale in which the user is looking at the catalog or display. Aswill be described in detail in this document, a user may wish to see animage of an item or scene altered to make the item look like it is inthe user's own ambient lighting, surrounds, lighting angle, and soforth. An accurate color rendering, however, is often desirable evenwhen it is not congruent.

As noted, the seller can use the physical color standard to alter thecaptured image (as the physical color standard is capable of beingcopied or standardized, and thus can be used to calibrate the imagecaptured with aid from a computer program or a person or both). Theseller can then provide the altered, improved image for the webpage orto a catalog printer. This is a partial solution, as it can improve theimage eventually rendered to the buyer. As noted, this is true for awebsite or color catalog or other rendering to show a more-accuratecolor for the image (here the blue-green shirt).

Second, and even if the seller does or does not alter the captured imagebased on the physical color standard, the seller can provide thisphysical color standard as captured with the image of the shirt or otheritem. Thus, in the eventual image rendered to the buyer, even at his orher own display (e.g., on a website on a buyer's smartphone or capturedon a buyer's camera of a catalog item), the physical color standard isshown or accessible. If the physical color standard is captured in theimage at the same time and/or under the same or similar conditions asthe item's capture, the ability to correct the image taken is higherthan if potential errors are introduced by capturing an image of thephysical color standard at some other time (though if done under thesame or similar ambient light and with a same or nearly sameimage-capture device, this can provide improved color rendering aswell).

In this example, with the image of the item and the physical colorstandard (e.g., in a same captured image), the techniques can correctthe image to accurately show the true color of the item. Here the truecolor includes at least a correct hue (e.g., red, green, blue), but thetrue color can also include a correct lightness, chroma, brightness,colorfulness, and saturation through corrections by the techniques. Todo so, an application, such as one on a buyer's computer, can comparethe physical color standard in the image with another physical colorstandard, such as one accessible by their rendering device or anaccurate physical copy of the physical color standard. The techniquesalter the rendered coloring of the item (e.g., hue, colorfulness,saturation, lightness, and/or brightness), and potentially other aspectsas well, based on corrections made to render the imaged physical colorstandard taken with the item to match or nearly match a physical colorstandard of the buyer, such as one in the buyer's own locale.

For example, an application on an image renderer's device (e.g., abuyer's phone) may alter the color of the image to match the physicalcolor standard, e.g., if the image's physical color standard is notproperly depicted (e.g., not accurate or not congruent). This iseffective to change the whole image of the item and standard until thephysical color standard as rendered on the buyer's phone matches thephysical color standard in the image renderer's locale, thereby alsocorrecting the item's color. When the imaged physical color standard isproperly depicted relative to the actual physical color standard, theitem's depiction will also be proper, as the color change can be done toboth the item and the imaged physical color standard at once (notrequired, but can be easier to perform than separately). Thus, assumethat the imaged item and physical color standard show a misbalance ofred/green or too little blue. The application can rebalance to match alocal physical color standard until the imaged physical color standardis properly depicted (e.g., looks the same to the viewer). When theimaged physical color standard is properly depicted, the item's colorwill also be properly depicted (or more so than previously).

Furthermore, the techniques can ascertain whether a user's display isshowing the correct color through knowledge of the display type, age,and so forth, as well as its settings. The techniques may then alter thedisplayed image based on matching to the correct physical color standardand the buyer's own device. This miscalibration of the buyer's devicecan also be corrected through the physical color standard by renderingan image of the physical color standard (e.g., with the device's owncamera or received) and then calibrating the display to match thestandard through device settings or a naked-eye comparison as notedherein. By so doing, the techniques enable a record of what changes tothe display settings should be made should a proper depiction of a sceneor item be desired. Note that many devices are intentionally calibratedto not be accurate or properly depict scenes, such as those with reducedblue light, or dimming for evening viewing, or altering displays toreduce power output.

In more detail, the techniques may also perform a calibration sequence,such as taking a picture of a color (here some or all of the physicalcolor standard), and then showing it to the user of the device andhaving the user calibrate the displayed color to match. Doing this a fewtimes with different colors and lightings can be sufficient for amore-accurate rendering of an image (even without needing to compare thephysical color standard for each rendering of an item or scene).

Adjusting for Ambient Color

The techniques disclosed herein also solve the problem of differentambient light (e.g., in color or angle). For example, even if acapturing entity accurately captures the image, and even if the image onanother entity's display is accurate, here meaning that it matches howthe item looked by a typical human eye when the item was imaged by thecapturing entity (e.g., the seller's own eye), the rendering of the itemby later-rendering entity, such as a buyer on the buyer's display, maynot be congruent with the buyer's current ambient conditions.

Consider a buyer looking at an item for sale on her smartphone. Assume,for this example, that the item is accurately rendered on the buyer'sdisplay. It may not match the situation in which the buyer is looking toplace the item, however. Return to the off-white chair example. Assumethat the light in which the item's image was captured was brightbluish-white light—e.g., non-natural light, or instead, that the item'simage was captured in natural-wavelength light but that the buyer'shouse is illuminated with incandescent (slightly yellow), fluorescent,or light-emitting diode (LED) light. In any of these cases, the buyer'ssmartphone's rendering will not properly depict the item with the colorthat it would have if the item had been in the actual ambient conditionsthat the buyer is in currently and/or where the buyer would like toplace the item (here the off-white chair). In such a case, properdepiction of the item is based on ambient conditions, rather than justcolor accuracy relative to the item or scene when it was captured.

The techniques can properly depict the rendered image on the buyer'sdisplay to show how the item would actually look in the buyer's locale.One way to do so is to sense, by a camera (e.g., a smartphone), theambient conditions (e.g., perform spectral analysis on ambient light forcolor factors, e.g., light wavelength and intensity). This can be donethrough the smartphone's camera or other sensors to determine thewavelength and color characteristics of the current, ambient light(e.g., wavelength, energy, etc.). With this information, the techniquesalter the rendering of the item on the smartphone such that therendering is altered to take into account the difference between theambient light in which the image was captured (if known), or at leastthe current ambient against the estimated/expected ambient lighting inwhich the image of the item was captured.

For example, if the item's image was captured in high-blue light, andthus the image is imbued with additional blue, the techniques can reducethe blue intensity for the rendered item. Similarly, if the ambient isfluorescent (assume here that it has a relatively poor color renderingindex, or CRI), the techniques can correct for the difference inspectrum (e.g., in some fluorescent light, reds in objects are shown toodimly). Or, if incandescent, to adjust to reduce yellow in the renderedimage (assuming a correlated color temperature, CCT, of 2700K). Whilesensors on the smartphone or other device can be used, as well asinformation about the item's image (known or assumed ambient for theitem during original capture or altered image thereafter) can be used,other manners disclosed here may also aid in altering a rendering tomatch ambient conditions.

If, however, the image for the item was captured with the physical colorstandard, and assuming that the rendering entity (e.g., buyer) has (orhad) the physical color standard, the current rendering of the image ofthe item can be altered such that the physical color standards match(the one captured with the item's original capture as well as the one ofthe buyer's). In so doing, the item can be rendered to be congruent withthe buyer's conditions, even if that congruency would be less accurate,strictly speaking, than some other rendering. For example, the renderingon the buyer's smartphone may accurately represent the item as it wascaptured, but that may not be what the buyer desires. The buyer maydesire to see what the item would look like if it were present, e.g.,seen in real life or at least rendered to match the ambient conditionssurrounding the buyer.

The techniques can make this adjustment automatically, based on anactual rendering of the physical color standard by the buyer's device(assuming that the buyer's camera and/or display are correctlycalibrated as noted herein). Or, the techniques can provide a userinterface whereby the user alters the rendered image of the item basedon seeing the item and the physical color standard captured with theitem along with another physical color standard, which the buyer canvisually compare to the imaged physical color standard (or automaticallyas noted herein). Thus, the user interface can present red, yellow, andblue hues, as well as saturation and brightness (and other aspects ofcolor), and the user can move each until the two standards match to thatbuyer's eye (these are but a few of many possible elements of color thatcan be selected to be changed through the techniques). The techniquesalso correct for differences in how people perceive colors, as the humaneye may not match, in some linear fashion, the actual technicalmeasurement of light.

Adjusting for Ambient Brightness and/or Luminosity

In addition to correcting for ambient hue, due to a difference in thehue of the ambient light versus the light in which the image wascaptured or altered and rendered, the techniques may also correct forambient brightness. Similarly, as noted above, the techniques permit anitem's image to be rendered to match the ambient brightness. Manyproducts for sale are captured in lighting that is very bright relativeto conditions in which a buyer would intend to use the item. Thus, theimage presented, whether accurately captured and rendered or not, is notcongruent with the buyer's current brightness. As above, the techniquesalso correct for brightness differences. One example includesdecorations, such as a pillow to be used on a buyer's existing couch.This pillow, if typical, was imaged in high-brightness and often with awhite background. Thus, the techniques may lower the brightness (andother measures of light) of the rendered image of the pillow such thatit is congruent with the ambient brightness. By so doing, the techniquesenable a rendering entity (here the buyer) to have a properly depictedrendering of the item or scene.

Augmented Reality

Continuing the pillow example above, assume that the pillow wasoriginally captured in a slightly too-blue ambient hue, a highbrightness, and on another couch not matching the buyer's brown couch.For a buyer to decide to buy the pillow, in conventional practice, thebuyer often has to 1) trust that the image was accurately captured, 2)trust that the image is accurately rendered by the buyer's device, 3)correct for ambient lighting differences (e.g., red, green, blue) in hisor her mind also to “guess how it would look,” and/or 4) correct forbrightness in his or her mind to “guess how it would look.” Thus, thebuyer would need to understand how the pillow would look with less blueand less brightness, as well as trusting that the image he or she seesis even accurate. Further still, many buyers would like to know how itwould look with their own décor, such as the brown couch. Even one ofthese challenges can be a problem for buyers, while two, three, four, orfive challenges, which is often the case, prohibits a good buyingexperience.

The techniques can also correct for one or even all of these fiveproblems, thereby permitting a more-accurate and/or more-congruentrendering, which, through a more-proper depiction of an item or scene,improving users' experience with catalogs, books, websites, and soforth. For example, a buyer's experience and confidence in his or herdecision to buy or not to buy an item can be improved.

The techniques can do so through use of augmented reality. In additionto, or alternatively to, one or more of the disclosed solutions, thetechniques can present an item's image superimposed over the buyer's ownconditions. Many smartphones can present real-time or nearly real-timerendering of a current capture of a user or the user's locale by thesmartphone's camera, e.g., in real time on the smartphone's display. Apreviously corrected-for image of an item can be shown in the display,superimposed over the rendered reality. This image for the item can bepreviously corrected as noted above, or it can be corrected within theaugmented reality interface. One way to do so is to use the physicalcolor standard, such as one that has some three-dimensionality to it(this can aid in lighting angle, permitting customization of the item'srendering to match the lighting angle or being able to select abest-angled image from multiple images of the item taken at differentlighting angles). Example physical color standards with 3Dcharacteristics are a cube, tetrahedron, sphere, semi-sphere, and soforth.

Continuing the above example, assume that the buyer is interested in asalmon-colored pillow for fall décor. The buyer, having a brown couch,wants to know how the pillow would actually look on the buyer's browncouch. With the augmented reality, the buyer uses a mobile device, suchas a tablet or smartphone, watches his or her own ambient and décorbeing shown, and then can move or have superimposed the item ofinterest, here the salmon-colored pillow, as it would look on thebuyer's couch. To do so, the item can already be corrected for, and theaugmented rendering of the local conditions shown accurately, and thenthe buyer has a good idea of how the item would look. However, many ofthe inaccuracies and incongruities of the image of the item can becorrected with the augment-reality technique. Assume that the buyer hasa physical color standard and places it on his or her couch. Then, thebuyer can correct the images presented on his or her display bycomparing, visually to the buyer's own eye, the imaged couch and itsphysical color standard in the display, with what the buyer sees withhis or her naked eye looking at the buyer's locale. With audioinstructions, a touch interface, or other manners, the buyer can adjustthe color (e.g., hue, brightness, colorfulness, saturation) of the couchas presented by matching his or her own matching of the naked-eye viewof the standard with the standard shown on the buyer's display. Thus,the buyer's locale will be rendered accurately on the buyer's device.Further, the standard need not be some international or consistentstandard as the same item being seen is also being rendered. Thus, auser may even be able to go without the standard by the techniquesenabling the user, through a user interface, to adjust the colors andbrightness so that the rendered couch matches the couch as seen by thebuyer's naked eye.

The item is superimposed and properly depicted to account for thecurrent conditions—this can be as noted above, or the item's image canbe altered to match through use of the physical color standard in theaugmented reality. Thus, a buyer can see the image of the item and thephysical color standard imaged with it and adjust the color, such asred, green, blue hues, brightness, lighting angle, saturation, so thatthe item is rendered much more congruently with how it would look in theactual room, the actual lighting hue, lighting brightness, and lightingangle.

With a congruent salmon-colored pillow on the augment display, alongwith the local conditions (the couch, lighting, etc.), the buyer canthen place the image of the pillow on the image of the couch, or thepillow can simply be centered or otherwise fixed, and the buyer can movethe device and/or camera so that the pillow is oriented as desired onthe couch. This augmented reality, in addition to the various benefitsalready noted, permits the buyer to “walk around” and get a better feelfor the item and how it fits. The techniques can alter the lightingangle of the item as the user walks around, assuming that the itemeither has multiple images for different lighting angles, or thetechniques can shade and highlight the item to approximate how the itemwould look with the light at the changed angle (as the buyer walksaround, steps left, right, back and so forth). The techniques can do soin part based on sensing lighting-angle differences as the user moves,e.g., for a three-dimensional physical color standard in the user'slocation.

While the example given is home furnishings, clothing and other itemscan also be properly depicted. Even without the physical color standard,the techniques enable a buyer to image, in augmented reality or viasnapshot, the color of the person's arm, for example, and then match therendered arm with how the arm looks to the buyer's naked eye. By sodoing, the rendering of the buyer's current conditions (arm, light, soforth) can be matched. Then, with the item's image made more congruentin any of the disclosed manners, the techniques present the item in thelocal conditions. Examples include how a scarf being sold on theinternet matches a person's favorite jacket, hair color, and skin tone.Other examples include how a shirt's color would match, clash, orcompliment a buyer's skin tone, hair, makeup color, and so forth.Further still, a makeup color can also be the item imaged, permittingmore-congruent rendering of makeup and therefore improved buyingdecisions.

The techniques permit better depictions of imaged scenes and items,thereby improving a user experience when viewing a website, catalog, orsocial media, for example. When a user is a buyer, his or her decisionsto buy can be improved. Consider, for example, use of a small, physicalcolor standard with some three-dimensionality. With makeup imaged, suchas lipstick, foundation, or rouge, and then using the techniques (withor without still-image or augmented-reality rendering), the item's colorand how that color would look on a particular person can bemore-accurately or congruently depicted. A makeup business, for example,could provide a foldable, small physical color standard with eachpurchase, or simply free online or in brick-and-mortar stores. Then,when a buyer would like to see how a catalog or online item would lookon him or her, the buyer folds the physical color standard into some 3Dshape and then uses the techniques to correct/make congruent themakeup's color and brightness and even lighting angle. The buyer maycompare and alter the image of the makeup and its accompanying physicalcolor standard to the buyer's own physical color standard, therebyaltering the image to be congruent with the conditions in which thebuyer's own physical color standard resides. Note that, by so doing,some purchases that would otherwise be performed in person can insteadbe performed remotely. This can especially aid buyers and sellers due tomobility limitations on many buyers, such as due to health concerns(e.g., the COVID-19 pandemic) or economic or ecological considerations,such as saving the environment or the buyer's resources by not drivingto a store.

Sizing

In addition to, or alternatively to, the techniques described above, animage of an item may also be correctly sized. While this can be done insome manners noted above, the techniques also permit doing so throughthe following manners. First, the techniques can use information aboutan item, such as its height, width, and depth, and correct the image(including the angle at which it is presented) and then associate thesize with the image. The size of the locale/conditions, e.g., thebuyer's body or décor, can be ascertained through direct entry ofdimensions or through use of a mobile device's ability to measure itemsin an image, such as Apple's® measure app, which can measure dimensionsof an item being rendered through augmented reality or a snapshot oflocal conditions, objects, and so forth. Rather than, or in addition to,these manners, the techniques may use the dimensions of the physicalcolor standard. Assuming that the physical color standard in an imageditem and the physical color standard is present at the buyer's locationare the same dimensions or that the difference in dimension is known,the techniques can scale up or down the image of the item for sale onthe webpage (or even paper catalog) so that it is correctly displayed inscale on the buyer's mobile device (e.g., the salmon-colored pillow willbe the correct size relative to the couch, thereby further improving thebuyer's decision making).

This can be especially useful for furniture, décor, jewelry, clothingaccessories, and clothing (when the clothing is imaged on amodel/mannequin, as many clothing items when presented flat or foldedare less useful for showing in scale). Assume that a buyer would like toknow if a particular bracelet would look good on her arm. The techniquespermit improved buying decisions for the buyer through improvedrendering of the item for sale. The bracelet can be shown over asnapshot or augmented, real-time image of the buyer's own wrist, in acongruent color, congruent brightness, congruent lighting angle, andcorrectly scaled to the buyer's own wrist. This is a substantialimprovement for buyers and sellers alike, even for non-website images,such as those in catalogs.

Example Environment

FIG. 1 illustrates an example system 100 in which techniques formore-accurate and/or more-congruent rendering of an imaged item can beembodied. System 100 includes a computing device 102, which isillustrated with four mobile examples: a laptop computer 102-1, a tabletcomputing device 102-2, a smartphone 102-3, and an electronic-bookreader 102-4, though other computing devices and systems, such asdesktop computers and netbooks, may also be used.

Computing device 102 includes computer processor(s) 104,computer-readable storage media 106 (media 106), display(s) 108, andinput mechanism(s) 110. Media 106 includes computer-executableinstructions that, when executed by the computer processor(s) 104,performed operations, such as those of an operating system 112 and animage module 114.

Image module 114 is capable of enabling or aiding techniques describedherein, such as improving the accuracy and/or the congruity of an imageat an image capture location (e.g., the disclosed seller) or an eventualimage-rendering location (e.g., the buyer).

Image module 114 may also include or have access to history 116, userinterface 118, and three-dimension module 120 (3D module 120). Userinterface 118 enables image module 114 to present, in user interface 118on display 108, the rendered images (e.g., current user locale in anaugmented reality with an item). The user interface 118 also permits,though input mechanisms 110, alteration by the user of the computingdevice 102 to alter a rendered image. 3D module 120 enables the imagemodule 114 to alter, in some cases, an image to show a different angleor lighting angle, and/or scale for an item, such as in an augmentedreality scenario. 3D module 120 can use a physical color standard withinan image, with an item and a physical color standard in the device'slocale to determine and alter a scale for the item. With use ofmeasurement sensors, alternatively, the 3D module 120 can determinedimensions for the locale and then scale the item's size appropriately.

The image module 114 can, for example, provide a user interface throughwhich to receive a user selection to alter a captured image of aphysical color standard, as noted further below. The image module 114receives the user selection and alters the captured images of thephysical color standard and an item or scene. The image module 114 mayalso or instead correlate a portion of an item or scene shown in animage to a matching color within the physical color standard where thematching color has a known location on the physical color standard (orif the location can be determined). By so doing, and based on therecorded color information for the known location, an actual color forthe portion of the item can be recorded. This enables a rendering to bemore accurate or more congruent. Further, a change to cause the color ofthe portion to match instead the recorded color information can also beapplied to all of the item, scene, or image, thereby improving theaccuracy or congruity of the entire item or scene.

History 116 can include the various data described herein, such asinformation about ambient light at a user's location, prior selectionsby the user (e.g., the buyer or seller), information about a currentdisplay (e.g., calibration data generated as noted herein), and evendata from other sources, such as other users' selections and displaydata.

Computing device 102 includes or has access to one or more displays 108and input mechanisms 110. Four example displays are illustrated in FIG.1 , all of which are integral with their respective device, though thisis not required. Input mechanisms 110 can include gesture-sensitivesensors and devices, such as touch-based sensors and movement-trackingsensors (e.g., camera-based), as well as mice (free-standing or integralwith a keyboard), track and touch pads, capacitive sensors (e.g., on asurface of computing device 102), and microphones with accompanyingvoice-recognition software, to name a few. Input mechanisms 110 may beseparate or integral with display 108; integral examples includegesture-sensitive displays with integrated touch-sensitive ormotion-sensitive sensors.

Imager 122 can include visible or non-visible light sensors, such ascameras, IR cameras (and projectors), and so forth. Imager 122 may alsosense ambient conditions, even without necessarily capturing an image.Further still, imager 122 can work with other components to provide theabove-noted augmented reality and other renderings.

Example Manual Change to Image

FIG. 2 illustrates an example in which the techniques enable a user toalter an image of an item. This alteration can correct, or makecongruent a rendered image of the item to local ambient conditions, orsimply correct an inaccuracy in the original image or the rendering ofthe image on the user's device.

As shown, the image module 114 presents an image 202 of an item 204along with an imaged physical color standard 206 with a user interface208 on a user device 210. The user interface 208 is configured toreceive a manual change to the image through any number of controls,from audio controls, gesture control, and so forth, such as wheelcontrols or slider-based controls 212 (here red, green, and blue hue,and brightness). The manual change is here based on a naked-eyecomparison of the imaged physical color standard 206 with a local,real-life physical color standard 214.

By way of example, consider FIG. 3 , which illustrates an initial set ofthe slider-based controls 212 and then two successive user selections,shown at 212-1 and 212-2. The user selection is shown with firstselection 302 and second selection 304. The first user selection is toreduce the blue hue, shown with the grey line 306 for the initial set ofcontrols 212, on the blue slider bar 308. The grey line is shownreducing the blue hue at reduced grey line 310. The second userselection further selects to reduce the blue hue, shown withfurther-reduced grey line 312. Note that the initial rendering of theimage 202 becomes less and less blue, going from a pinkish-blue salmoncolor to an orange-salmon color. Note also that the physical colorstandard 206 is altered concurrent with the item (though this can bedone prior to the item's color change). The original rendering of thephysical color standard becomes less blue and then further less blue,shown at 314 and 316, respectively. Here note also that the real-lifephysical color standard 214 now matches the less blue standard 316.Thus, as the standard rendered more-closely matches the real-lifestandard, the item 204 also becomes more-closely congruent and/oraccurate with the user's locale, shown at improved rendering 318 andfurther-improved rendering 320.

Thus, the image module 114 receives the manual change through the userinterface 208 and then changes the image 202 based on the receivedmanual change. This image, once altered, is rendered or saved for lateruse, though this alteration can be in real time, delayed, or simplyprovided to another interface, such as an augment-reality interface 216shown in FIG. 10 . Note that this user interface 208 can be augmentedreality (presenting a color image of the user's locale along with theimaged item and physical color standard) or an interface that does notpresent the user's locale in the interface.

Example Method for Color Rendering

Below is an example method 400 (illustrated in FIG. 4 ) by which thetechniques improve an accuracy or congruity of a current or futurerendering of an item or scene (e.g., anything that can be imaged). Forall methods disclosed herein, the operations may be reorganized (e.g.,in a different order than shown), iterated, repeated, and combined, ifthe context allows. The ordering shown is not required except as setforth herein.

At 402, a user interface is provided through which to receive a userselection to alter a captured image of a physical color standard.Example user interfaces are shown in FIG. 2 , FIG. 3 (in part), FIG. 5 ,FIG. 6 , and FIG. 9 , as well as the description of the techniquesabove. In FIGS. 2, 3 , and 9, for example, user interface 208 providesslider-based controls 212 by which to alter a color of an image.

Also as noted herein, the method can provide a user interface withmultiple selectable controls, each of the multiple selectable controlspresented with a different representation of the physical colorstandard. This can aid a user in selecting quickly or iteratively.Consider, for example, FIG. 5 , which illustrates a range of fivedifferent brightness choices though a user interface 502 (this is oneexample of the user interface 118) on the smartphone 102-3.

At 404, the user selection to alter the captured image of the physicalcolor standard is received through the user interface. Examples ofselections through a user interface are described above, such as a userinterface that enables reception of a user selection through a usercontrol that, through manual selection, changes a color of the capturedimage of the physical color standard. As noted herein, the change incolor can be a change to alter a hue, colorfulness, saturation,lightness, or brightness of the captured image of the physical colorstandard. A manual selection to change a color is illustrated in FIG. 3and described above, with a user selecting to reduce a blue hue of acaptured image.

Continuing the example of FIG. 5 , the user interface 502 enablesselection of a broad range (gross range) at brightness 504-1 through504-5, such as with a simple tap gesture on the display, of one of thefive presented images of the physical color standard, each showing adifferent brightness. As noted, FIG. 5 also shows the physical colorstandard 506 in a user's locale. Each of these operations can beiterated, re-performed, altered and performed, and so forth. An exampleof this is also shown in FIG. 5 , where finer gradations of selection(e.g., fine-range brightness 510, with five finer-gradations ofselectable brightness, 510-1, 510-2, 510-3, 510-4, and 510-5) or withdifferent categories of color alterations, e.g., after selecting abrightness, the techniques may present differently hued options,different saturation, and so forth, but with a brightness at or near theselected brightness.

At 406, the altered captured image of the physical color standard ispresented through the user interface or another user interface and basedon the received user selection. Many examples of this are describedherein, such as at FIG. 3 , which illustrates the altered, capturedimage of the physical color standard 314 and then further altered at316, showing less blue than 314 and 206. Continuing the example of FIG.5 , selection of one of the brightness ranges, here 504-3, results inpresentation of fine-grade alterations (as well as a reproduction of theselected brightness 504-3) of the physical color standard, shown at510-1, 510-2, 510-4, and 510-5. While not required to be performed bythe method or a particular element described above, some entity altersthe image prior to the presentation at 406, such as a remote entity orimage module 114.

At 408, an image of an item or scene captured in a same locale as thatin which the captured image of the physical color standard was capturedis altered. This can be altered based on a received selection ornumerous received selections, such as those described in FIG. 3 or FIG.5 , to alter the captured image of the physical color standard capturedin the same locale as the item or scene. This alteration of the image ofthe item or scene, and therefore a more-accurate or more-congruentrendering of the item or scene, can be performed separately, together,or contemporaneously. In the example shown in FIG. 3 , the image module114 alters the captured image of the physical color standard and theimage of the item or scene in one contemporaneous operation, such aswhen the item or scene is captured in a same image as the physical colorstandard. Thus, as the physical color standard is altered in FIG. 3 ,the item is also altered, both of which are presented in the userinterface 208 (not shown in FIG. 3 for visual brevity).

At 410, the altered image of the item or scene is recorded or presented.Presentation of the altered image of the item or scene is shown in FIG.3 at 320. The alteration can also or instead be recorded, therebyrecording the change in color caused by the alteration of the physicalcolor standard. This record of the change is effective to enable afuture change to a future image to cause the future image tomore-accurately or more-congruently be presented. Thus, if the image ofthe item or scene is sent to another viewer along with the record, theimage or scene can be rendered more accurately based on that record. Ifthe eventual display used to render the item or scene is similar insettings and/or display type, this further aids in the rendering be moreaccurate or more congruent. Further alterations can also be made, asnoted herein.

If the record, or a combination of records, shows changes to imagesselected by the user for a display associated with the user, the recordcan later be used to automatically correct a future-received or capturedimage. This is a form of automatic alteration based on what is acalibration of the user's display or, if the user is capturing theimage, a calibration of the user's imager 122 and the display 108. Thiscan be saved in history 116 of FIG. 1 . By so doing, the method 400 mayautomatically alter another image of another item or scene where theother image of the other item or scene captured in the same locale. Thisautomatic alteration is based on a difference between the alteredcaptured image of the physical color standard and the captured image ofthe physical color standard, or a user selection as noted herein.

While the physical color standard is illustrated as a 3D color box witha broad range of hues, other physical color standards can be used. Forexample, if the item or scene is a makeup product, the physical colorstandard may include a range of human skin tones at higher resolution orwith a smaller physical size of the standard. Other examples include useof a physical color standard with higher-resolution human hair and skintones, brightness, or even saturation than the illustrated physicalcolor standard. Further still, if the item is a decoration item, adifferent standard for in-home ambient conditions can be used. If thescene is a picture of a person, to gain a more-accurate rendering, thephysical color standard may include the hair, skin tones, and range ofclothing colors. If the scene is a landscape picture, the range ofcolors present in outdoor locales can be represented in the physicalcolor standard.

As noted in part above with the description of FIG. 5 , the techniquescan provide multiple selectable controls presented with a differentrepresentation of the physical color standard. Generally, at 412,different representations of the physical color standard are determined.These can be determined in various manners, including through thefollowing operations. At 414, a portion of the captured image of thephysical color standard is correlated to a matching color within apreviously recorded image of the physical color standard or a copy ofthe physical color standard, the matching color having a known locationon the previously recorded image of the physical color standard or thecopy of the physical color standard. The matching color can be amatching hue, colorfulness, saturation, lightness, or brightness. At416, based on recorded color information for the known location, anactual color for the portion of the captured image of the physical colorstandard is determined. And, at 418, based on the actual color for theportion, the different representations of the physical color standardare determined, such as multiple representations that are of a finerrange of color options, thereby enabling a faster or more-preciseselection. Thus, the image module 114 is able to present what are likelyto be closer matches and better selectable choices.

For example, consider FIG. 6 , which illustrates a range of ninedifferent hues and levels of brightness presented through a userinterface 602 (this is but one example of the user interface 118) on thesmartphone 102-3. Here, the user interface 602 enables a selection ofnine color/brightness ranges 604, such as with a simple tap gesture onthe display, of one of the nine presented images of the physical colorstandard, each showing a different color and brightness. As shown, theleft column has a higher red, the right higher blue, the upper rowhigher brightness, and so forth. In these examples, selectable anddifferent representations are determined as shown in the alternativeoperation 412, resulting in the nine different representations, each ofwhich is based on determining an actual color and then presentingoptions that are near to, or surrounding that actual color, such asthrough a similar hue balance, brightness, saturation (not shown), andso forth.

As noted above, the methods herein can be repeated, iterated, and soforth. For example, on selection by a user, either to select differentcharacteristics or even multiple characteristics at one interface (e.g.,a grid of brightness and hue shown in FIG. 6 ), the image is altereduntil the user is happy with the match to the physical color standard(e.g., 506). As noted, the user may instead or in addition manuallyalter the color through the interface (e.g., audio input to “increasebrightness and reduce red,” or through slider controls 212, and soforth).

Returning to the operation 412, instead of performing the operations414, 416, and 418, alternatively or in addition, the operation 412determines the different representations through a prior-performedcalibration. This calibration can be performed explicitly based on ahuman-selectable comparison to calibrate a display on which the userinterface is presented or can be based on the record in which one ormore prior changes to the color of an image of a physical color standardis rendered on a same or similar display. One example is where the imagemodule 114, with imager 122, captures an image of a physical colorstandard in a user's locale. Then, using parts of the method 400, theuser manually alters the rendering to match what the user sees of thereal-life physical color standard. In so doing, the alteration is a formof calibration of the display. In some cases, however, the alterationmay take into account inaccuracies in the imager 122 and may thereforebe a less-than-perfect calibration.

Example Method for Color Rendering Through Color Matching

Below is an example method 700 (illustrated in FIG. 7 ) by which thetechniques improve, through color matching, an accuracy or congruity ofa current or future rendering of an item or scene. For all methodsdisclosed herein, the operations may be reorganized (e.g., in adifferent order than shown), iterated, repeated, and combined, if thecontext allows. The ordering shown is not required except as set forthherein. Note that the method 700 can be performed without userinteraction, in whole or in part, and thus can be automaticallyperformed, such as by the image module 114 of FIG. 1 , except where anexplicit user interaction is described.

At 702, an image of a physical color standard is received. The physicalcolor standard has a known color in a known location within the physicalcolor standard. This physical color standard can be one that iscalibrated or otherwise known to be accurate, as noted at accuratestandard 800 in FIG. 8 , or data that represents the standard in amachine-usable format. FIG. 8 illustrates a known color 802 and location804 on the accurate standard 800. This does not have to be shown to auser; it can instead be non-visual and/or performed solely by a computerprocess (e.g., the image module 114 having instructions to cause theprocessor 104 to perform the operations of method 700) using the imageor a data file having the known location and known color. The locationcan be cartesian or otherwise.

The known color 802 is known for at least a hue and may also be knownfor other characteristics of color. The location 804 is also known andcorrelated to the known color 802. While not required, the known color802 in the known location 804, within the standard 800, can bedetermined through a calibration, the calibration based on ahuman-selectable comparison to calibrate a display on which the userinterface is presented.

At 704, an image of an item or scene is received. The image of the itemor scene and the image of the physical color standard are a same image,images captured contemporaneously, or images captured in a same locale,such as a same location or a same ambient condition.

At 706, a portion of the item or scene shown in the image of the item orscene is correlated to a matching color within the physical colorstandard, the matching color having the known location on the physicalcolor standard. The physical color standard in one captured in a same orsimilar locale, such as within a same image as the item or scene. Thecorrelation of the portion of the item can include sampling multipleportions of the image of the item or scene with known colors of thephysical color standard or vice versa, and where the portion correlatedis responsive to a match based on the sampling. Further, thiscorrelation can be to match a hue, colorfulness, saturation, lightness,or brightness of the portions.

Consider, by way of example, FIG. 8 , which illustrates the image 202 ofthe item 204 of FIG. 2 , along with the physical color standard 206within the same image 202. At 706, the image module 114 correlates aportion 806 of the item 204, the portion 806 having a matching color tothat of a portion 808 of the image of the physical color standard 206.Note that the color rendered on the image of the physical color standard206 may or may not be correctly rendered, but if the item 204 and theimage of the physical color standard 206 are both incorrectly renderedin a similar manner, the item's color can be corrected.

At 708, based on recorded color information for the known location, anactual color for the portion of the item or scene is determined. Herethe portion 808 is at a location 810, the location 810 on the image ofthe physical color standard 206 mapped to the location 804 of theaccurate standard 800, which is correlated to the known color 802. Theknown color 802 is the actual color that the portion 806 of the item 204should be rendered.

At 710, the recorded color information and a location of the portion ofthe item or scene are recorded. This recording is effective to enable afuture rendering of the image of the item or scene to correctlyrepresent an accurate or congruent color, in a future rendering, of theportion of the item. In the illustrated example, this recorded colorinformation is the known color 802 and the portion 806 of the item 204.

At 712, the techniques may render, based on the recorded colorinformation and the location of the portion of the item or scene, theimage of the item or scene having the accurate or congruent color. Whilenot required, this rendering of the item or scene having the accurate orcongruent color enables a user to check this accuracy or congruity andalter it according to various manners noted above (e.g., operation 404of method 400).

Method for Color Rendering Using Recorded Color Information

Below is an example method 900 (illustrated in FIG. 9 ) by which thetechniques improve a rendering of an item or scene to be more accurateor congruent. For all methods disclosed herein, the operations may bereorganized (e.g., in a different order than shown), iterated, repeated,and combined, if the context allows. The ordering shown is not requiredexcept as set forth herein. Note that the method 900 can be performedwithout user interaction, in whole or in part, and thus can beautomatically performed, such as by the image module 114 of FIG. 1 ,except where an explicit user interaction is described.

At 902, an image of an item or scene is received. While not required,the recorded color information and the location of the portion of theitem or scene can result from operation of the method 700. In such acase, the image can be received from a remote device that performed themethod 700 or a same device as performing the method 900 if the method700 was performed on the same user device (e.g., the computing device102).

At 904, recorded color information and a location of a portion of theitem or scene are received. The recorded color information and thelocation of the portion of the item or scene indicate an accurate orcongruent color of the portion of the item or scene at the location. Therecorded color information and the location indicate the accurate colorof the portion of the item or scene at the location, rather than acongruent one to a user's ambient condition, and further includereceiving, through a user interface, a user selection to further alterthe rendering of the altered image of the item or scene. In such a case,the further alteration can be based on a local physical color standardin a locale viewable by a user, the further alteration effective toimprove a congruity of the altered image of the item or scene to ambientlighting conditions of the locale.

At 906, a difference in color between a color of the portion of the itemor scene in the image and the recorded color information is determined.

At 908, the image of the item or scene is altered based on thedetermined recorded color or the difference in color. Note that thechange to this portion's color to match the recorded color can beperformed on just the portion or all of the item or scene, or anywherein between. Thus, a change to reduce a red hue and increase a saturationof a portion of an item to match a recorded color can be applied to morethan the portion. This enables a change to be made without in-depthanalysis of each portion of the item or scene. In this document, theterm “alter” is used to mean various changes to an image, includingproducing a new image, altering a file for an image, and then using thealtered image, and so forth. The term alter is not intended to requireuse of the same image or data file for the image, as a new image or datafile can instead be created, in which case the altered image is the newimage.

At 910, the altered image of the item or scene is recorded or rendered.As noted above, the rendering of the altered image can be on the display108, using a user interface 118, on computing device 102. The userinterface, however, can be an augmented reality interface, which the 3Dmodule 120 can produce.

As noted in part above, the techniques enable calibration of a displayto better depict an image. In such a case, a further alteration to theimage can be performed automatically based on the calibration. Or thealteration can be received manually through a user selection tocalibrate a display on which the altered image is rendered. This furtheralteration improves a congruity or accuracy of the altered image of theitem or scene when rendered on the display.

Example Augmented Reality

FIG. 10 illustrates but one example of an augmented-reality interface.Here, the augmented-reality interface 216 through which the item 204,the imaged physical color standard 206 (optionally, as prior correctionmay have been performed, e.g., see FIG. 2 ), and an imaged locale 218,which represents a physical locale 220, can be presented. Note that theimage 202 of the item 204 and the physical color standard 206 aresuperimposed over the imaged locale 218. This example also includes theslider-based controls 212 through which a user may alter the brightnessand color of the imaged locale 218 (and, separately, the item 204, whichmay have been previously corrected using the techniques). As notedabove, a local, physical color standard 222 can be imaged in theaugmented-reality interface 216 as an augmented-reality image 224. Theaugmented-reality image 224 can be altered until it matches the user'snaked-eye perception of the local, physical color standard 222. In sodoing, the item 204 as imaged and the locale 220 as imaged (at 218) canbe congruent with the user's locale 220. Thus, the buyer may see if thesalmon-colored pillow for sale matches his or her décor (here, the couchand table of the physical locale 220). Thus, the techniques permitalteration to images of an item for sale and the locale in which it isintended to be placed to correct inaccuracies and/or incongruities.While not shown, the user may instead alter the image 202 and the item204 and place the user's display on the couch, which permits congruentcoloring, but may make a correct size of the item 204 challenging (mostpillows are larger than smartphone and tablet displays).

Additional Seller and Buyer Examples

Below is an example method 1100 (illustrated in FIG. 11 ) by which thetechniques enable a seller to provide a more-accurately rendered imageof an item, such as one for sale. For all methods disclosed herein, thesteps may be combined, reorganized (e.g., in a different order thanshown), iterated, repeated, and combined. The ordering shown is notrequired except as set forth herein.

At 1102, an image of the item is captured, along with a physical colorstandard. This can be performed in any of the above-noted manners, suchas by a user with the imager 122.

At 1104, an application (e.g., the image module 114) receives thecaptured image of the item and the physical color standard.

At 1106, the application compares the imaged physical color standard inthe captured image with a recorded image for the physical colorstandard, or data usable to assign an accuracy to, the imaged physicalcolor standard.

At 1108, the application alters the image of the item and the physicalcolor standard based on the comparison. The alteration of the image iseffective to improve the accuracy of the image of the item. Note thatthis improved accuracy can be an accuracy for the ambient color in whichthe image is captured, thereby correcting camera error in rendering,and/or it can correct the image to a different ambient color, such as arecorded image of the physical color standard taken in a preferredambient lighting (e.g., white light from the sun).

Alternatively, or in addition, at 1110, the application may present theitem in a user interface enabling the seller to use his or her naked eyeto adjust the image to match what the seller sees (e.g., the item asimaged to the physically present item or the physical color standard asimaged to the physically present physical color standard). At 1112, theapplication receives a manual change to the image (e.g., based on theuser's naked-eye comparison of the physical color standard in the imagewith a locally present physical color standard). In so doing, theapplication (e.g., image module 114) alters the image of the itemresponsive to the user's input, then records the altered image at 1114.This can be done after, before, or in conjunction with other operationsof the method.

Alternatively, or in addition, at 1116, the techniques can aid a sellerin selecting multiple altered physical color standards by which to aidthe seller in correcting the image of the item. As noted later in thecontext of FIGS. 5, 6, and 12 , the seller may be similarly aided by thetechniques to select a better-matching image of the physical colorstandard and thereby a more-accurate image of the associated item. Thealtered physical color standards are determined at 1116, presented at1118, and selection is received at 1120. Ways in which these images canbe determined, selected, and ways in which these steps can be repeatedand arranged are set forth similarly in the description of FIG. 12 andillustrated in FIGS. 5 and 6 . For example, the seller's own history ofalterations in the past on his or her device or camerasettings/preferences can be known and used to aid the techniques toprovide altered images likely to be near the accurate, naked-eye view ofthe physical color standard, or a broader range can simply be presentedand selected iteratively to determine an accurate alteration.

Returning to 1114, the altered image and/or input made to alter theimage is recorded or provided, such as in the display as described inthe user interface 208 of FIG. 2 (and FIG. 3 ), the augmented-realityinterface of FIG. 10 , and the user interface 502 of FIG. 5 .

Example Method, Buyer Side

Below is an example method 1200 (illustrated in FIG. 12 ) by which thetechniques enable a buyer to more-accurately or more-congruently renderan image of an electronic format (email, text, social media, webpage,etc.) or a printed item or scene, such as one for sale. For all methodsdisclosed herein, the steps may be combined, reorganized (e.g., in adifferent order than shown), iterated, repeated, and combined. Theordering shown is not required except as set forth herein.

A simplified example method 1200 (illustrated in FIG. 12 ) is providedbelow by which the techniques enable a more-accurate or more-congruentimage of an electronic (email, text, social media, webpage, etc.) or aprinted item for sale.

At 1202, a captured image is received by an application. This capturedimage can be already improved as noted for the seller-side method, or itcan be unimproved. In this method (but not some other manners describedherein), the image is captured of the item along with a physical colorstandard. For printed images, such as catalogs and advertisements, thereceived image can be from a buyer's own device by capturing an image(e.g., taking a picture) of the printed image having the item andphysical color standard.

As one option, at 1204, the application compares the imaged physicalcolor standard in the captured image with a locally imaged copy of asame type of physical color standard as that imaged with the item (e.g.,a copy). The comparison of the captured image to the locally imagedimage for the physical color standard (e.g., by the image module 114)can be used to determine, for the ambient light in which the locallyimaged image was captured, differences between the two, and thenautomatically adjust the image of the item and physical color standard.This improves an accuracy and/or a congruity of the color, such as hue,brightness, lightness, and saturation of the image for the item.Examples include reducing brightness to match a lower-light ambient,altering a red, green, or blue hue to be more or less present, and soforth. The techniques, at this operation, can use historical data toimprove the image of the item as noted herein.

At 1206, the application alters the image of the item based on thecomparison. At 1208, the image of the item is provided, such as to adevice of the buyer. Note that if the device's display has flaws inaccurate rendering, these can be corrected, or the user can compare therendered, altered image to a physical color standard and, through his orher naked eye and a provided user interface, alter the image of the itemfurther with a manual change, at 1212 and 1214 (or performed as part of1206 and 1208).

Note that the method can be iteratively performed, or performed in realtime and repetitively, such as when the ambient conditions are changingor the buyer's perspective for the physical color standard is changing.

Also, in some cases, an application on the seller side can perform aspectral analysis of the light when the image was captured, and this canbe used by the application on the buyer side (which may be a same ordifferent application) to aid in correcting the image of the item.

Also, in some cases, the techniques can use knowledge of the buyer'sdisplay's type, age, and flaws to help in altering the image, such asprior to any optional adjustment by the buyer of the image for the item.Furthermore, the image module 114 may enable the viewer (e.g., thebuyer), such as previous to the method, to use the techniques tocalibrate his or her display using an image of a physical color standard(received or taken by the buyer), and then alter the display'scalibration through a user interface and with a naked eye looking at alocal physical color standard. This can improve the display generally,or can provide calibration data so that, when an image is received thatmay be of import to the buyer for color (e.g., an item for sale), thecalibration can be used by the techniques to improve the rendering ofthe item for sale (e.g., automatically at 1216 and 1218 or other methodoperations disclosed herein). This can be done with other manners orseparately, such as doing so first, and then the techniques asking thebuyer to alter the image through the user interface to furtherimprove/alter the image.

Alternatively, or in addition, at 1216, the techniques determinemultiple altered physical color standards. These may be determined usingthe many mentioned manners, such as device settings (for example, yellow“night time” coloring present for some displays, known characteristicsof the device, e.g., reduces blue to save eye strain, and so forth,user-set preferences for device hue), user history (e.g., priorselection by a user indicating that the device's display renders imagesas imbalanced color, e.g., chroma, hue, luminosity), seller-associatedhistory (e.g., seller is known to image items in a particular ambientlight, etc.), and even current ambient conditions through sensors (e.g.,a dark room, fluorescent or LED lighting, and so forth). On these bases,1216 determines some range of hues (or other characteristics). These canbe a final or first step in aiding the buyer to select a physical colorstandard imaged on the device that is close to the naked-eye physicalcolor standard present at the buyer's locale. In some cases, however, abroad range is determined, which may include little or no analysis, butinstead provide a broad range of choices for selection by the buyer,which then can be narrowed down (if the buyer would like furthergranularity, accuracy, and/or congruity) based on the prior selection.

At 1218, the techniques cause some number of these determined renderingsof the altered physical color standard to be presented for selection.Consider, for example, FIG. 5 , which illustrates a range of fivedifferent brightness choices though a user interface 502 (this is oneexample of the user interface 118) on the smartphone 102-3. Here, theuser interface 502 enables selection of a broad range (gross range) atbrightness 504-1 through 504-5, such as with a simple tap gesture on thedisplay, of one of the five presented images of the physical colorstandard, each showing a different brightness.

At 1220, the techniques receive selection of one of the determinedrenderings of the altered physical color standard. Continuing theexample, the user interface 502 receives a selection of one of thebrightness 504 selectable images. Assume for FIG. 5 that the buyerselects brightness 504-3 as most-closely matching the buyer's naked-eyejudgment against the physical color standard 506 shown in the buyer'slocale 508. At this point, operations 1216, 1218, 1220, and 1208 can beiterated one or more times, but with finer gradations of selection(e.g., fine-range brightness 510, with five finer-gradations ofselectable brightness) or with different selections, e.g., afterselecting a brightness, the techniques may present differently coloredoptions, with different hue balance (or other characteristics) but witha brightness at or near the selected brightness.

For example, consider again FIG. 6 , which illustrates a range of ninedifferent hue/brightness range presented through a user interface 602(but one example of the user interface 118) on the smartphone 102-3.Here, the user interface 602 enables selection of nine hue/brightnessranges 604, such as with a simple tap gesture on the display, of one ofthe nine presented images of the physical color standard, each showing adifferent hue and brightness. As shown, the left column has a higherred, the right higher blue, the upper row higher brightness, and soforth.

This can be continued, either to select different characteristics oreven multiple characteristics at one interface (e.g., a grid ofbrightness and hue) until the buyer is happy with the match to thephysical color standard (e.g., 214, 222, 506). The buyer may insteadmanually alter the characteristics through an interface (e.g., audioinput to “increase brightness and reduce red,” or through slidercontrols 212, and so forth).

Returning to the method of FIG. 12 , at 1210, the techniques optionallyrecord any manual changes to hue or brightness input by the buyer to thebuyer's device and/or the alteration. This can aid in further improvinglater performance of the application for future items. Thus, if a buyeralters an image, the application can learn what to change in the future,such as a buyer's selection to increase red in an image and correlatethis increase in red to data about the image of the item and data aboutambient conditions. Through this ongoing record of changes from a buyerto images, the application can learn how to better alter a further imagefor rendering on the buyer's display. This same data can be provided toother instances of the application or received from other instances andused to improve the automatic corrections by the application. Forexample, if another buyer alters a blue hue a particular amount for animage of an item (the same or different item) when the ambient light isof a particular character at the other buyer's device, this can berecorded and passed to/used by the buyer's application to automaticallyalter the image if the ambient lighting at the buyer's device is similar(or the contrast between the ambient light during the original captureof the image is similar). This data, however, can be optionally sharedby the buyer's choice.

Additional Disclosure

The functionalities provided by the techniques enable capture of animage of a scene along with an established physical color standard(e.g., a color wheel, color bar, 3D physical color standard, having tensto even millions of colors, as the human eye can distinguish about 10million different colors, etc.) and generate an image with true,representative colors. For example, when oriented for a provider of animage, such as a seller of an item online or through a catalog, theimage module 114 enables the image of the item to be adjusted tomore-closely match the true or congruent hue and luminosity of the item,as well as other factors.

To do so, the image module 114 calculates a difference between thecaptured image's physical color standard values and correct physicalcolor standard values stored or accessible by the computing device 102.For example, assume a scene is captured containing a desired product andthe physical color standard. The techniques can analyze the image andidentify the physical color standard. The colors on the physical colorstandard represented in the image can vary depending on circumstantialfactors that were present when capturing the image. For example, thescene may have been poorly lit, or the computing device's Auto-WhiteBalance (AWB) functions may have been skewed. The colors represented inthe image of the physical color standard can be compared to those of anestablished physical color standard stored within or accessible by anapplication configured to perform the techniques (e.g., the image module114). The comparison will produce the difference between the hue values.Alternatively or in addition, the difference is known or determined froma rendering application or even rendering hardware, such as a graphicsprocessing unit (GPU), which includes information about the color foreach pixel rendered in the image. By comparing information about thecolor for pixels of an image of a physical color standard that has beencorrected against pixels to render an image of a capture image of aphysical color standard, the difference in color characteristics at therendering level is determined. Note that these pixels can be correlated,such as by mapping in cartesian coordinates, between portions of each ofthese standards.

For example, the physical color standard in the image may contain a redhue value of R231 at a particular location (or pixel), while theestablished (or corrected) physical color standard on the applicationmay designate that corresponding red hue at that location should insteadbe R220. Thus, the application would produce a difference of −11 asbeing the true red hue in the image (and thus the item or scene aswell). The application may produce hue differences for all the physicalcolor standard values until a representative image can be generated.Alternatively, true-hue bars and luminosities can be displayed in realtime, and the user can adjust the hues per these bars. Even further, aspectral analysis of the light could be taken (e.g., with a lightsensor) to ensure that the hues, brightness, and so forth are correct orcongruent.

The functionalities provided by the techniques on thepotential-buyer-end permit the user to visualize a more-accurate and/orcongruent representation of an item's color. The techniques can retrievethe device's display metrics (e.g., screen type, age, settings) andanalyze the physical color standard provided by the seller. By comparingthe device's display metrics and the physical color standard provided bythe seller, the techniques can then accurately portray the image'scolors. For example, the image module 114 may determine the device'sblue hue is displayed on a lower intensity than its red or green hue.Thus, an application configured to perform the techniques can calculateand display the image with respect to the device's display settings andthe physical color standard provided by the seller. Alternatively, theapplication can determine the screen color through a calibrationsequence, such as taking a picture of a color, displaying it, and havingthe user calibrate the displayed color to match. The calibrationsequence can be repeated until representative colors are depicted. Thiscalibration can be used to alter an image in methods described herein.Additionally, the application on a wireless-communication device (e.g.,a smartphone, a tablet) permits the user to capture an image of thescreen where an item (e.g., a product for sale) is displayed. Theapplication on the wireless-communication device can alter the capturedimage based on the physical color standard provided with the image andthe established physical color standard within the application (orviewed by the buyer instead or after initial correction by theapplication).

Note that color has many characteristics, such as hue, colorfulness,saturation, lightness, and brightness, to name a few. Hue, sometimes acolor appearance parameter, can be defined based on a degree ofstimulus, or a single number related to a color space coordinate diagramor color wheel (See FIG. 2 ). Or hue can be defined based on a dominantwavelength or that of its complementary color. Colorfulness andsaturation are attributes of perceived color relating to chromaticintensity. These are formally defined by the International Commission onIllumination, though this formal definition is not required. In moredetail, colorfulness can depend both on spectral reflectance andstrength of illumination. Saturation is the colorfulness of an areajudged in proportion to its brightness. Brightness is an attribute ofcolor as well and is associated with a visual perception of radiating orreflecting light. Thus, a low-luminance object is not very bright.Lightness is sometimes referred to as a colors value or tone, and is arepresentation of brightness. Various models of lightness are present inthe art, from Munsell and the HSL or HSV color models. As will beapparent to one skilled in the art of color, the characteristics ofcolor are sometimes overlapping and often are based on perception of auser, which can vary from some users to others. This variance, however,can be addressed through the disclosed techniques to create a congruentimage of an item or scene to a particular user, even if that congruencymay not appear as congruent to another person.

Also, as noted above and described in more detail in this section, thetechniques can use one or many of the characteristics of color to alterand improve rendering of color images.

EXAMPLES

Below are provided various examples.

Example one: A method for improving an accuracy or congruity of an imageshowing an item, the method comprising:

-   -   capturing an image of an item along with a physical color        standard;    -   comparing the image of the physical color standard to a recorded        image standard;    -   altering the image of the item based on the comparison of the        image of the physical color standard to the recorded image        standard; and    -   recording the altered image.

Example two: the method of example one, further comprising:

-   -   presenting the altered image of the item along with the physical        color standard with a user interface configured to receive a        manual change to the image, the manual change based on a        naked-eye comparison of the imaged physical color standard with        a local, real-life physical color standard;    -   receiving the manual change through the user interface; and    -   further altering the altered image of the item based on the        received manual change, and where recording the altered image        records the altered image with the manual change.

Example three: one or more computer-readable media having instructionstherein that, responsive to execution by one or more processors, performoperations of the method of examples one or two.

Example four: a mobile computing device having:

-   -   a display;    -   one or more computer processors; and    -   one or more computer-readable media having instructions there        that, responsive to execution by the one or more processors,        perform the operations of the method of examples one or two and        further renders the altered image on the display.

Example five: a mobile computing device having means for performing theoperations of examples one, two, or three.

Example six: a method for improving an accuracy or congruity of an imageshowing an item, the method comprising:

-   -   receiving an image of an item along with a physical color        standard;    -   comparing the image of the physical color standard to a        locally-imaged copy of the physical color standard;    -   altering the image of the item based on the comparison of the        image of the physical color standard to the locally-imaged copy        of the physical color standard; and    -   providing the altered image for rendering.

Example seven: a method for improving an accuracy or congruity of animage showing an item, the method comprising:

-   -   receiving an image of an item along with a physical color        standard;    -   presenting the image of the item along with the physical color        standard with a user interface configured to receive a manual        change to the image, the manual change based on a naked-eye        comparison of the imaged physical color standard with a local,        real-life physical color standard;    -   receiving the manual change through the user interface; and    -   providing the altered image for storage or rendering.

Example eight: one or more computer-readable media having instructionstherein that, responsive to execution by one or more processors, performthe operations of the method of example six or seven.

Example nine: a mobile computing device having:

-   -   a display;    -   one or more computer processors; and    -   one or more computer-readable media having instructions there        that, responsive to execution by the one or more processors,        perform the operations of the method of examples six or seven        and further comprising rendering the altered image on the        display.

Example ten: a mobile computing device having means for performing theoperations of examples six, seven, or eight.

CONCLUSION

Although aspects of color rendering have been described in languagespecific to features and/or methods, the subject of the appended claimsis not necessarily limited to the specific features or methodsdescribed. Rather, the specific features and methods are disclosed asexample implementations of color rendering. Accordingly, otherequivalent features and methods are intended to be within the scope ofthe appended claims. Further, various different aspects are described,and it is to be appreciated that each described aspect can beimplemented independently or in connection with one or more otherdescribed aspects.

The invention claimed is:
 1. A method comprising: providing, on a mobiledisplay, a user interface through which to receive a user selection toalter a captured image of a physical color standard, the captured imageof the physical color standard captured in a first ambient light, theuser interface configured to receive a change to the captured image ofthe physical color standard based on a naked-eye comparison of thecaptured image of the physical color standard with a local, real-lifephysical color standard; receiving, through the user interface, the userselection to alter the captured image of the physical color standardbased on the naked-eye comparison of the captured image of the physicalcolor standard with the local, real-life physical color standard, thelocal real-life physical color standard in a second ambient light, thesecond ambient light different from the first ambient light; presenting,on the mobile display and through the user interface or another userinterface and based on the received user selection, an altered capturedimage of the physical color standard, the altered captured image of thephysical color standard enabled, through the user selection, to be moreaccurate or congruent with the second ambient light than the capturedimaged of the physical color standard; altering, based on the receiveduser selection or the alteration to the captured image of the physicalcolor standard, an image of an item or scene captured in a same localeas that in which the captured image of the physical color standard wascaptured; and presenting, within an augmented reality interface and inreal-time or nearly real-time, the altered image of the item or scene,the altered image of the item or scene being more accurate or congruentwith the second ambient light than the image of the item or scene. 2.The method of claim 1, wherein providing the user interface providesmultiple selectable controls, each of the multiple selectable controlspresented with a different representation of the physical colorstandard, and wherein receiving the user selection includes selection ofone of the multiple selectable controls.
 3. The method of claim 2,further comprising determining the different representations of thephysical color standard, the determining comprising: correlating a firstportion of the captured image of the physical color standard to a secondportion of a previously recorded image of the physical color standard ora copy of the physical color standard, the first or second portionhaving a known location on the captured image of the physical colorstandard or the previously recorded image of the physical color standardor the copy of the physical color standard, respectively; determining,based on recorded color information for the known location, an actualcolor for the portion of the captured image of the physical colorstandard; and determining, based on the actual color for the portion,the different representations of the physical color standard.
 4. Themethod of claim 3, wherein the matching color is a matching hue,colorfulness, saturation, lightness, or brightness.
 5. The method ofclaim 3, wherein the previously recorded image of the physical colorstandard or the copy of the physical color standard is determinedthrough a calibration.
 6. The method of claim 1, wherein the userinterface enables reception of the user selection through a user controlthat, through manual selection, changes a color of the captured image ofthe physical color standard, and wherein receiving the user selection toalter the captured image of the physical color standard includesalteration of a hue, colorfulness, saturation, lightness, or brightnessof the captured image of the physical color standard.
 7. The method ofclaim 1, wherein: the method performs operations including the alteringthe image of the item or scene and the presenting the altered image ofthe item or scene; the captured image of the physical color standard andthe image of the item or scene are within a same captured image; andaltering the captured image of the physical color standard and alteringthe image of the item or a scene are a same, contemporaneous operation.8. The method of claim 1, further comprising automatically alteringanother image of another item or scene, the other image of the otheritem or scene captured in the same locale, the automatic alterationbased on: a difference between, or alteration of, the altered capturedimage of the physical color standard and the captured image of thephysical color standard; or the user selection.
 9. The method of claim1, wherein the item or scene is a makeup product.
 10. The method ofclaim 3, wherein correlating the first or second portion includessampling multiple portions of the captured image of the physical colorstandard with known colors of the physical color standard, and whereinthe first or second portion correlated is responsive to a match based onthe sampling.
 11. The method of claim 3, wherein correlating the firstor second portion matches a hue, colorfulness, saturation, lightness, orbrightness.
 12. The method of claim 3, wherein the correlating matches acolor having the known location through a calibration, the calibrationbased on a human-selectable comparison to calibrate the mobile display,the human-selectable comparison based on the naked-eye comparison or aprior naked-eye comparison of the captured image of the physical colorstandard with the local, real-life physical color standard.
 13. Themethod of claim 1, wherein presenting the altered image of the item orscene within the augmented reality interface presents the item within auser's locale.
 14. The method of claim 1, wherein presenting the alteredimage of the item or scene though the augmented reality interfacepresents the altered item or scene with an image of objects within alocale viewable by a user, the image of the objects including an imageof the local, real-life version of the physical color standard.
 15. Themethod of claim 14, further comprising further altering the alteredimage of the item or scene based on a difference between the capturedimage of the physical color standard and the image of the local,real-life physical color standard.
 16. The method of claim 15, whereinthe further altering the altered image of the item or scene is performedautomatically and without requiring a user selection.
 17. The method ofclaim 1, further comprising saving a calibration for the mobile display,the calibration based on the received user selection or the alterationto the captured image of the physical color standard, the calibrationusable to alter a future image for presentation on the mobile display,and further comprising receiving the future image and automaticallyaltering, based on the calibration, the future image to provide analtered future image, the altered future image more accurate or morecongruent than the future image prior to the alteration.
 18. The methodof claim 1, wherein the local, real-life physical color standard and thecaptured image of the physical color standard are both a same face,hand, arm, or hair of the user.
 19. The method of claim 1, furthercomprising: receiving a second image of a second item or scene;receiving recorded color information or a first location on the first ora second captured image of a second physical color standard by which toattain the recorded color information and a second location of a portionof the second item or scene, the recorded color information indicatingan accurate or congruent color of the portion of the second item orscene at the second location; determining a difference in color betweena color of the portion of the second item or scene in the image of thesecond item or scene and the recorded color information; altering thesecond image of the item or scene based on the determined difference incolor; and recording or rendering the altered second image of the itemor scene.
 20. The method of claim 19, wherein the recorded colorinformation or the first location indicate an accurate color of theportion of the second item or scene at the second location, and furthercomprising receiving a second user selection to further alter therendering of the altered second image of the item or scene, the furtheralteration based on the local, real-life physical color standard beingin a locale viewable by a user, the further alteration effective toimprove a congruity of the altered second image of the item or scene toambient lighting conditions of the locale.
 21. The method of claim 1,further comprising, prior to receiving the user selection: altering aprior image of the item or scene and a prior image of the physical colorstandard, the altering effective to provide the image of the item orscene or the image of the physical color standard, respectively, forpresentation in the user interface, the altering the prior imagesincluding: receiving the prior image of the physical color standard, thephysical color standard having a known color in a known location withinthe physical color standard; receiving the prior image of the item orscene captured in the same locale as that in which the prior image ofthe physical color standard was captured; receiving or determining acorrelation correlating a portion of the item or scene shown in theprior image of the item or scene to the known location on the physicalcolor standard; and determining, based on the known color, an alterationeffective to improve an accuracy of a color of the prior image of thephysical color standard or the prior image of the item or scene.
 22. Themethod of claim 9, wherein presenting the altered image of the item orscene within the augmented reality interface presents, with a user'shair, skin, or face, the makeup product, a portion of the makeupproduct, or a color of the makeup product.
 23. The method of claim 22,wherein presenting the altered image of the item or scene within theaugmented reality interface presents the makeup product, the portion ofthe makeup product, or the color of the makeup product superimposed overat least a portion of the user's hair, skin, or face.
 24. The method ofclaim 22, wherein presenting the altered image of the item or scenewithin the augmented reality interface presents the makeup product, theportion of the makeup product, or the color of the makeup product at alighting angle matching the user's hair, skin, or face.
 25. The methodof claim 13, wherein presenting the altered image of the item or scenewithin the augmented reality interface presents the item superimposedover at least a portion of the user's locale.
 26. The method of claim25, wherein presenting the altered image of the item or scene within theaugmented reality interface presents the item to match a lighting angleof the user's locale.
 27. The method of claim 25, wherein the user'slocale includes a hair, skin, face, or clothing of the user, and whereinthe item is clothing, jewelry, or a clothing accessory.